Compressing Probability Distributions

Manipulating probability distributions is central to data compression, so it is natural to ask how well we can compress probability distributions themselves. For example, this is useful for probabilistic reasoning [2] and query optimization [6]. Our interest in it stems from designing single-round asymmetric communication protocols [1,4]. Suppose a server with high bandwidth wants to help a client with low bandwidth send it a message; the server knows the distribution from which the message is drawn but the client does not. If the distribution compresses well, then the server can just send that; conversely, if the server can help the client in just one round of communication and without sending too many bits, then the distribution compresses well — we can view the server’s transmission as encoding it.